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Abstract — This work proposes a causal and recursive algorithm 
for solving the "robust" principal components' analysis (PCA) 
problem. We primarily focus on robustness to correlated outliers. 
In recent work, we proposed a new way to look at this problem 
and showed how a key part of its solution strategy involves 
solving a noisy compressive sensing (CS) problem. However, 
if the support size of the outliers becomes too large, for a 
given dimension of the current PC space, then the number of 
"measurements" available for CS may become too small. In 
this work, we show how to address this issue by utilizing the 
correlation of the outliers to predict their support at the current 
time; and using this as "partial support knowledge" for solving 
Modified-CS instead of CS. 



I. Introduction 

In this work, we propose a real-time (causal and recursive) 
algorithm for solving the "robust" principal components' anal- 
ysis (PCA) problem. Here, "robust" refers to robustness to 
both independent and correlated sparse outliers, although we 
focus on the latter. The goal of PCA is to find the principal 
components' (PC) space, which is the smallest dimensional 
subspace that spans (or, in practice, approximately spans) a 
given dataset. Computing the PCs in the presence of outliers 
is called robust PCA. If the PC space changes over time, there 
is a need to update the PCs. Doing this recursively is referred 
to as "recursive robust PCA". Often the data vectors (both the 
low rank component and the outliers) are correlated. 

A key application where this problem occurs is in automatic 
video surveillance where the goal is to separate a slowly 
changing background from moving foreground objects. The 
background sequence is well modeled by a low rank subspace 
that can gradually change over time, while the moving fore- 
ground objects constitute the "correlated sparse outliers". We 
will use this as the motivating problem in this entire paper. 
Other important applications include sensor networks based 
detection and tracking of abnormal events such as forest fires 
or oil spills; or online detection of brain activation patterns 
from fMRI sequences (the "active" part of the brain can be 
interpreted as a correlated sparse outlier). 

Note that even though we use the term "outlier" for the 
moving objects or the brain active regions etc, quite often, 
these are actually the "signals of interest" . But for PCA (used 
to estimate the low rank subspace of the background), they are 
the "outliers" that make the PCA problem difficult. 

The recursive robust PCA problem, that we study in 
this work, can be precisely defined as follows. The outlier- 
corrupted data vectors (we will call them measurements), M t , 



at each time t, can be rewritten as 



M t = L t + S t 



(1) 



where L t is the "low-rank" part and St is the sparse outlier 
which can be correlated over time. We put "low-rank" in 
quotes since it does not mean anything for a single vector. 
To make this precise, we can rewrite 

L t = Ux t 

where U is an unknown orthonormal matrix and x t is a sparse 
vector whose support changes slowly over time and whose 
elements are spatially uncorrected. Let N t denote the support 
of x t , and assume that it is piecewise constant with time. Then 
the columns of the sub-matrix, P t := (U)jsr t span the low 
rank subspace in which the current set of L t 's lie and L t = 
Pt(%t)N t - We refer to P t as the principal components (PC) 
matrix. Clearly, this is also piecewise constant with time. If at 
a given time, t = to, N t changes, then P t also changes. New 
directions get added to it and some directions get removed 
from it. We assume realistic correlation models on both x t 
(and hence L t ) and on St. These are defined in Sec. HH 

Suppose, we have a good estimate of the initial PC matri^Q, 
Pq « Pq. For t > 0, our goal is to recursively keep estimating 
the sparse part, St, at each time, and to keep updating P t every - 
so-often, by collecting the recent estimates of L t = M t — S t . 

In (Q}, if U were known, and if we did not want a recursive 
solution, then this problem would be similar to the dense error 
correction problem studied in recent work [JT], [0. Of course 
the reason we need to do PCA is because U is unknown. 

There has been a large amount of earlier work on robust 
PCA e.g. 0, H and recursive robust PCA e.g. 0, @. In 
these works, (i) either the locations of the missing/corruped 
data points are assumed known [5], which is not a practical 
assumption, or (ii) they first try to detect the corrupted pixels 
and then either fill in the corrupted location using some 
heuristics or (iii) often just remove the entire outlier vector 
El , El, (6). Very often, outliers affect only a small part 
of the data vector, e.g., the moving objects may occupy 
only a small image region. In a series of recent works Q, 
0, 0, an elegant solution has been proposed, that treats 
the outliers as sparse vectors and hence does not require 
a two step outlier location detection/correction process and 
also does not throw out the entire vector. In 0, 0, (9], 

most applications, it is easy to get a short sequence with no moving 
objects that can serve as the "training data" to estimate Pq by simple PCA. 



the offline "robust PC A" problem is redefined as a problem 
of separating a low rank matrix, L := [Li, . . . : L t ], from 
a sparse matrix, S := [Si, . . . , St], using the data matrix, 
M := [Mi, . . . , M t ] = L + S. It was shown that by solving 
min||L||* + A||S||i, s-*- M = L + S (where ||L||* is the sum 
of singular values of L while ||5|| i is the t\ norm of 5 seen as 
a long vector), one can recover L and 5 exactly provided the 
singular vectors of L are spread out enough (not sparse), the 
support and signs of S are uniformly random (thus it is not 
low rank) and the rank of L is sufficiently small for a given 
sparsity of S. This was called Principal Components' Pursuit 
(PCP). PCP was motivated as a tool for video surveillance 
applications to track moving objects (sparse part) from the 
slow changing background (low rank part). 

While PCP is an elegant idea, it has three practical limi- 
tations. In surveillance applications, one would like to obtain 
the estimates on-the-fly and quickly as a new frame comes in, 
rather than in a batch fashion. Second, in many applications, 
the support sets over time will be heavily correlated, and often, 
also overlapping, e.g. in the case of moving objects forming 
the sparse part. This can result in S being low rank and thus 
makes it impossible for PCP to separate S from L [see Fig. 
. Third, PCP requires the support of x t to be fixed and quite 
small for a given support size of St, e.g. see Table 1 of Hi. 
But, often, this does not hold. 

To address the first two drawbacks, in [10], we proposed 
a recursive robust PCP (RRPCP) algorithm that was also 
"robust" to time-correlated sparse outlier sequences. In this 
work, we show how we can use the time -correlation of the 
"outliers" to our advantage to address the third limitation 
above. 

A. Motivation and Key Ideas 

The key idea of Recursive Robust PCP (RRPCP) OH is 
as follows. Assume that the current PC matrix P t has been 
estimated. We project the outlier-corrupted data vector, M t , 
into the space perpendicular to it, i.e. we compute 

y t := A t M u where A t := (P t ,_L)' 

and P tj ± is an orthogonal complement of P t . If M t is 
n dimensional, then the dimension of y t is n — r where 
r := rank(Pt). Notice that 

y t = A t S t + p t , Pt '= A t L t = A t Pt(xt) N f 

If P t « P t , then A t P t « 0, i.e. this should nullify most 
of the contribution of the low rank part, so that f3 t can be 
interpreted as small "noise". Finding the sparse outlier, St, 
from this projected data vector, y t , now becomes a noisy 
sparse reconstruction / compressive sensing (CS) flTTI . [Q2L 
lf]~3l problem. We can solve 

min ||s||i subject to \\y t - A t s\\ 2 < e (2) 

s 

with e chosen as explained in ifTOl . Denote its output by St. 
We can then estimate L t = M t — S t which can be used to 
recursively update P t every- so-of ten as in [lOl Algorithm 1]. 
When a new direction gets added to P t , but P t is not yet 
updated, the "noise", ft, will start increasing. We explain how 



to "cancel" some of it in Sec. III-BI after explaining the model 
on L t . The block diagram is shown in Fig. |l(a)[ Our idea is 
also somewhat related to that of lfT4l . ifHl in that all of these 
also try to cancel the "low rank" part by projecting the original 
data vector into the perpendicular space of the tall matrix that 
spans the "low rank" part. But the big difference is that in all 
these, this matrix is known. In our problem this matrix (the 
PC matrix) is unknown and can change with time. 

Now, if the support size of the sparse outliers increases 
for a given rank r, the number of projected "measurements" 
available for the CS step, n — r, may become too small for 
CS to work. But notice that the correlated sparse outliers 
(e.g. moving foreground objects in video) can be interpreted 
as sparse signal sequences whose support change over time 
is either very slow; or quite often is not slow, but is still 
highly correlated, e.g. the support can "move" or "expand" 
or change according to some model, over time. By using 
a model on how the objects move/deform, or other models 
on how the support changes, it should be possible to obtain 
a support prediction that can serve as an accurate "partial 
support knowledge". We can then tap into our recent work 
on Modified-CS which solves the sparse recovery problem 
when partial support knowledge is available [16). Denote the 
partial support knowledge by T. Modified-CS tries to find the 
solution that is sparsest outside the set T among all solutions 
that satisfy the data constraint. It does this by replacing ||s|| 1 in 
© by || st c ||i- As proved and experimentally shown in flTTI . 
in the noisy case, it has stable and small error using much 
fewer measurements than what simple CS needs. 

In this work, we demonstrate the above idea by assuming 
that the foreground contains a single rigid moving object 
that follows a constant velocity model (with small random 
acceleration). We use a Kalman filter (KF) to track its motion 
over time. The KF predicted location of the object and its 
size tell us its predicted support at the current time. This is 
then used to solve Modified-CS and obtain an updated support 
estimate. The centroid (or median) of this support set tells us 
the observed location of the object, which may be erroneous 
because our support estimate is not perfect. This serves as the 
noisy observation for the KF update step. 

B. Notation 

The set operations U, D and \ have the usual meanings. For 
any set T C {1, • • -n}, T c denotes its complement and |T| 
denotes its cardinality. 

For a matrix A, we let A4 denote the i th column of A and we 
let At denote a matrix composed of the columns of A indexed 
by T. We use A! to denote transpose, and A^ to denote its 
pseudoinverse. For a tall matrix A, A^ = (A' A)~ l A' . 

For vector v, V{ denotes the zth entry of v and vt denotes a 
vector consisting of the entries of v indexed by T. We \\v\\k to 
denote the Ik norm of v. The support of v, supp(v), is the set of 
indices at which v has nonzero value, supp(v) := {i : Vi ^ 0}. 

C. Paper Organization 

We define the problem and give the correlation models in 
Sec. HI Our proposed method, Support-Predicted Modified-CS 
RRPCP (SuppPred-ModCS-RRPCP), is described in Sec. [Till 
where we also argue why it should be stable. Experiments 
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(a) Recursive Robust PCP 



(b) Support-Predicted Modified-CS 



Fig. 1: Block Diagram of SuppPred-ModCS-RRPCP 



showing that SuppPred-ModCS-RRPCP provide a significant 
gain than RRPCP OH and PCP |7] are given in Sec.HVl 

II. Problem Definition and Correlated Models 
We give the problem definition below and explain the corre- 
lation models on L t and St in Sec. III-BI and III-CI respectively. 

A. Problem Definition 

Consider the problem of tracking moving foreground objects 
(correlated sparse outliers) in a slowly changing background 
(low rank part). For explaining our ideas in a simple fashion, 
we assume a ID "image" with one moving foreground object. 
The extension to 2D image and multiple moving objects is 
explained in Sec. JV1 Our simulations use this case. 

In video applications, the outlier (moving object) is not 
added, but is overlaid. In other words, at each t, the image 
M t satisfies 

'{O t )i ieT t 
\Lt)i ieTi 

where O t is the sparse foreground image and T t is its support 
i.e. (Ot)r t c = 0. and L t is the background image. However 
this problem can be rewritten in the form (Q3 by letting 

(a 





(M t )i 



(3) 



L t )i 



Notice that S t and O t have the same support T t . 



ieT t 



(4) 



B. Model on L t (background image) 

For L t , we use the model from |[T0l . Briefly, we assume 
that the support of x t , N t , is piecewise constant with time, and 
also assume that each nonzero element of x t is independent 
and follows a piecewise stationary model with nonstationary 
transients between pieces. Specifically, an independent first 
order autoregressive (AR) model is assumed for each nonzero 
element of x t , all with the same AR parameter < / < 1 
as given in ifTUl . Every d frames, there are some indices get 
added or deleted from N t . When a new direction added to P t , 
initially x t along that direction starts with some initial small 
value, but slowly increases to a stable large value. Before an 
existing direction gets deleted from P t , x t along that direction 
decays exponentially to zero. 

When new directions are added to P t , but are not part of the 
current P t , the "noise", f3 t , will start to increase. But by using 
the AR model, we can cancel some of the noise by replacing 
y t with y t ,canc defined as 



Now, the "noise" is only f3 t — f$t- 
than fit if / is close to 1 and if L t - 



i which is much smaller 

i « Lt-i. 



C. Model on Support Change of St (foreground image) 

Let p t be the location of the foreground object's centroid, 
let v t denote its velocity and let w denote its width. Thus, its 
support is, 

T t = [p t - w,p t + w] 



Let 



9t := 



and F := 



1 1 
1 



We assume a constant velocity model with small random 
acceleration [|T8l on the object's motion, i.e., 



9t = Fg> 



t-i 



(6) 



where n t is bounded noise with zero mean and variance Q. 
The variance matrix Q is of the form 



q 



because we only 



add noise to velocity and not to position ifTSIl . 

III. Support-Predicted Modified-CS Recursive 

Robust Principal Components' Pursuit 
In this section, we explain our algorithm. 

A. Support-Predicted Modified-CS 

Recall that in RRPCP, the number of projected "measure- 
ments" available for the CS step is n — r where r = rank(P t ). 
If the support size of the sparse part, \T t \, increases for a 
given r, then n — r may become too small for CS to work. 
In this section, we show how to use the correlated support 
change of St along with Modified-CS to address this problem. 
The overall idea is as follows. The support of St follows 
the model given Sec. III-CI We use this in a KF to obtain 
its location prediction, which can then give us its support 
prediction T t \t-i> We then solve Modified-CS, given in (fT4l) . 
with T = T t \t-\. The support estimate of the Modified-CS 
output serves as the updated support, T t \t> an d the centroic@ of 
this updated support serves as the "observed" location, pt,obs, 
for the KF update step to update the object's location estimate. 
We explain each of these steps below. 

Predict Centroid: Let g t \ z = [pt\ z v t \ z ]' represent the 
estimate of g t at time t given measurements up to, and 
including at time z. Similar rule applies for T t \ z . Let E t i t _i, 



yt,< 



y t - fA t L t -i 



(5) 



One can replace the centroid by the median for more robustness. 
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Algorithm 1 Support-Predicted Modified-CS 

1) Predict centroid by (Qj) and (Q2]> 

2) Predict support by (f]~3T> 

3) Update support 

- Modified-CS: solve d) using T = f t . 

- Add-LS-Del procedure: 

T add = TU{ieT c : \(S)i\ >a add } (7) 

(s) Tadd = ((A t ) Tadd yyt,canc, (s) T o dd = (8) 

T t \ t =T add \{ieT add : \(s)i\ <a del } (9) 

(S t ) f =((A t ) f Jy t , canc , (S t ) fc =0 (10) 

* i * * i * ±\t 

4) Update centroid by ([16]), <[T7J and JT51 . 



E t | t and if t denote the prediction and updated error co variance 
matrices and the Kalman gain used by the KF. Compute 

g t \t-i = F g t -i\ t -i (11) 
X t \t-i = F E t _i| t _i F' + Q (12) 

Predict Support: We can get a reliable support prediction as 

T t \t-i = [Pt\t-i ~ ™,Vt\t-i + w] (13) 

Update Support using Modified-CS: Assuming T t \t-i is a 
good support prediction, we can use it as the partial support 
knowledge for Modified-CS, i.e. we can solve 

min||sT<=||i subject to \\y ti canc - A t s\\ 2 < e (14) 

s 

with T = T t | t _!. Let s be the solution of (O with T = f t \ t _ 1 . 
As explained in ifTTlL since s is biased towards zero along T c 
and it may be biased away from zero along T (there is no cost 
or constraint on s^), we will run into problems if we try to use 
a single threshold for support estimation. A better approach 
is the use the Add-LS-Del procedure as summarized in step 
3 of Algorithm [T] This was first introduced in our older work 
ED, E3 and simultaneously also in EQ, E2). It involves 
a support addition step (that uses a smaller threshold), as in 
©, followed by LS estimation on the new support estimate, 
T add , as in ([8]), and then a deletion step that thresholds the 
LS estimate, as in ©. This can be followed by a second LS 
estimation using the final support estimate, as in ([TOt . The 
addition step threshold, a add , needs to be just large enough to 
ensure that the matrix used for LS estimation, AT add is well- 
conditioned. If a add is chosen properly, the LS estimate on 
T add will have smaller error than the Modified-CS output. As 
a result, deletion will be more accurate when done using this 
estimate. This means that one can also use a larger a de i to 
ensure quicker deletion of extras. 

Update Centroid: Let pt,obs denote the "observed" centroid 
of foreground object at time t obtained by taking the mean of 
the updated support estimate T t \ t , i.e., pt y0 bs = centroid(T t \t)- 
Our observation model is 

Pt,obs = Hg t +uj t (15) 

where H := [10] and uo t is observation error, which is 
assumed to be zero mean with variance R. This arises because 



Algorithm 2 SuppPred-ModCS-RRPCP 

At t = to, suppose a good estimate of PC matrix, P is 
available from training data. For t > to, do the following: 

1) Update P t using Algorithm 1 in [10] which is based on 
||5l and correspondingly, update A t := • 

2) Obtain y t , C anc by ©. 

3) Support-Predict Modified-CS using Algorithm [T] 

4) Estimate 6 t and L t by (j3 and (l20l 

5) Increment t by 1 and go to step 1). 



there are extras and misses in T t \ t and hence pt )0 bs = 
centroid(T t \ t ) ^ centroid(T t ) = pt- In this work, R is 
heuristically selected, but, in general, one can approximate it 
using simplifying assumptions on the support computation. 
The KF update step is as follows. 

K t = E t | t _! H' (H E t | t _x H' + R)- 1 (16) 

9t\t=9t\t-i + K t (pt,obs ~ H g t \t-i) (17) 

^t\t = ^t\t-i - Kt H £*|t-i (18) 

The above steps for Support-Predicted Modified-CS are 
summarized in Algorithm [T] and in block diagram of Fig. |l(b)| 

B. Complete Algorithm of SuppPred-ModCS-RRPCP 

With the support estimate T t \ t and sparse estimate St 
obtained in Algorithm [T] Ot and L t can be estimated as 

(^)f t|t =(^)f t|t , (00^=0 (19) 
L t =M t -St (20) 

Also, P t can be updated as in [10, Algorithm 1]. 

A complete algorithm incorporating the idea of Support- 
Predict Modified-CS is given in Algorithm [2] 

For simplicity, we have presented SuppPred-ModCS- 
RRPCP for a ID image with one moving foreground object. 
However, it can be easily extended to the 2D case with 
multiple moving objects. We explain how to do it in Sec. [iVl 

C. Discussion 

SuppPred-ModCS-RRPCP is a recursive approach. Hence 
an important question is when and why will it be stable? We 
try to give an induction argument here that we will formalize 
in later work. The key idea is as follows. Everywhere in 
this discussion, "bounded" means bounded by a time-invariant 
value. Suppose that at t — 1, \pt-i — Pt— lit— 1 1 is bounded 
and small. Since n t is bounded and small, this means that 
\pt — Pt\t-i\ is also bounded and small. This, in turn will 
mean that the same holds for the support prediction errors 
|Tt\T t i t _ 1 | and \T t \t-i\T t \. Using this and arguments similar 
to those in ifTTl . the support update step will also result in T t | t 
with bounded and small extras and misses (in fact the bound 
on extras is zero). This step will require showing that the 
"noise" seen by modified-CS, f3 t — fA t L t -i, is bounded; that 
most nonzero elements of St are large enough; and that A t 
satisfies certain conditions. Finally, since T t \ t has zero extras, 
we will just need to argue that the misses, T t \ T t \t, will 
result in bounded and small centroid observation error, uj t . 




Fig. 2: Images at time t = 1,5, 100. 

This will finally ensure bounded and small \p t — p t \ t \. This, 
along with ensuring stability of E t | t , will ensure bounded and 
small \p t —Pt\t\- Our simulations given in Sec.|lV]do indicate 
that SuppPred-ModCS-RRPCP is stable. 

We use the KF in this paper, but in general the above 
argument will go through with any stable linear observer. 

IV. Numerical Experiments 
We evaluate the performance of SuppPred-ModCS-RRPCP 
in the 2D case for the problem of tracking two moving objects 
in a simulated image sequence of size 28 x 28 x 100. Figj2] 
shows the image frame at t = 1, 5, 100. One image written as 
a ID vector, M t , is n dimensional with n = 28 2 = 784 and 
it satisfies ©. 

The background image, L t is simulated according to the 
model in Sec jII-B[ Initially, there are 350 principal directions 
in the PC matrix P t . At t = 5, one new direction is add to P t 
with a small variance and it slowly stabilizes. At t = 30, one 
existing direction starts to decay exponentially to zero. Thus, 
r := rank(P t ) w 350. 

The foreground overlay, Ot, consists of two 11x11 blocks 
that have different constant intensity 80 and 50. These two ob- 
jects move independently. Each object moves along horizonal 
and vertical directions independently with some initial location 
and velocity satisfying ©.In © and (TBI) , we use zero 
mean truncated gaussian noise with variance q = 10 -4 and 
R = 10 -3 for n t and uo t . Note that \T t \ := \supp(S t )\ ~ 242, 
while n — r is only about 434. 

Fig. [3] shows the reconstruction error of S t for two online 
methods, Algorithm [2] and RRPCP (10||, as well as offline PCP 
0. For offline PCP, we use the entire sequence and show the 
error for each image frame separately. PCP fails due to the 
correlation of St. The other two online methods work well 
because they do not require the support and signs of St to 
be i.i.d. Algorithm [2] outperforms RRPCP greatly because it 
utilizes the correlation model of St while RRPCP does not. 

With an estimate of T t , we separate different objects by 
thresholding the intensity of M t on the support estimate. This 
is needed for running two separate KFs for each of their 
centroids. 

We show the number of extras and misses in T t \ t -i and T t \t 
when using Algorithm [2] in Fig. HI Recall that Tt\t-i is the 
predicted support used by Modfied-CS (O, and T t \t * s the 
updated support estimate obtained by Modfied-CS followed 
by Add-LS-Del 0-([[Ol). Clearly, this corrects a lot of the 
prediction errors. 

References 

[1] John Wright and Yi Ma, "Dense error correction via 11 -minimization", 
IEEE Transactions on Information Theory, 2009. 

[2] J.N. Laska, M.A. Davenport, and R.G. Baraniuk, "Exact signal recovery 
from sparsely corrupted measurements through the pursuit of justice", 
in Asilomar Conf. on Sig. Sys. Comp., Nov 2009, pp. 1556 -1560. 




Fig. 3: Comparison of Algorithm 12 RRPCP, and PCP 



|Tt\T t | 



l r t\T t | t | .a,.. |T t | t _AT t | .\T t \ t \T t \ 



\Tt\ 



\Tt\ 



\Tt\ 



\Tt\ 




Fig. 4: Support error of Support-Predicted Modified-CS 

[3] F. De La Torre and M. Black, "A framework for robust subspace 

learning", in ICCV, 2003, pp. 54:117-142. 
[4] H. Xu, C. Caramanis, and S. Mannor, "Principal component analysis 

with contaminated data: The high dimensional case", ArXiv e-prints, 

May 2010. 

[5] M. Brand, "Incremental singular value decomposition of uncertain data 

with missing values", in ECCV, 2002, pp. 707-720. 
[6] D. Skocaj and A. Leonardis, "Weighted and robust incremental method 

for subspace learning", in IEEE Conf. on Comp. Vis. Pat. Rec. (CVPR), 

Oct. 2003, pp. 1494 -1501 vol.2. 
[7] E. J. Candes, X. Li, Y. Ma, and J. Wright, "Robust principal component 

analysis?", Submitted for publication. 
[8] V. Chandrasekaran, S. Sanghavi, P. A. Parrilo, and A. S. Willsky, "Sparse 

and low-rank matrix decompositions", in Allerton, 2009. 
[9] A. Ganesh, J. Wright, X. Li, E. J. Candes, and Y. Ma, "Dense Error 

Correction for Low-Rank Matrices via Principal Component Pursuit", 

ArXiv e-prints, January 2010. 
[10] C. Qiu and N. Vaswani, "Real-time robust principal components' 

pursuit", Allerton, 2010. 
[11] Scott Shaobing Chen, David L. Donoho, and Michael A. Saunders, 

"Atomic decomposition by basis pursuit", SIAM lournal on Scientific 

Computing, vol. 20, pp. 33-61, 1998. 
[12] E. Candes, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact 

signal reconstruction from highly incomplete frequency information", 

IEEE Trans. Info. Th., vol. 52(2), pp. 489-509, February 2006. 
[13] D. Donoho, "Compressed sensing", IEEE Trans, on Information Theory, 

vol. 52(4), pp. 1289-1306, April 2006. 
[14] E. Candes and T. Tao, "Decoding by linear programming", IEEE Trans. 

Info. Th., vol. 51(12), pp. 4203 - 4215, Dec. 2005. 
[15] Y. Jin and B. Rao, "Algorithms for robust linear regression by exploiting 

the connection to sparse signal recovery", in 1CASSP, 2010. 
[16] N. Vaswani and W. Lu, "Modified-cs: Modifying compressive sensing 

for problems with partially known support", IEEE Trans. Signal 

Processing, Sep 2010. 
[17] N. Vaswani, "Stability (over time) of modified-cs and ls-cs for recursive 

causal sparse reconstruction", Allerton, 2010. 
[18] H. Vincent Poor, An Introduction to Signal Detection and Estimation, 

Springer, second edition. 
[19] N. Vaswani, "Ls-cs-residual (ls-cs): Compressive sensing on least 

squares residual", IEEE Trans. Sig. Proc, vol. vol. 58(8), pp. 41084120, 

2010. 

[20] N. Vaswani, "Kalman filtered compressed sensing", in IEEE Intl. Conf. 

Image Proc. (ICIP), 2008. 
[21] W. Dai and O. Milenkovic, "Subspace pursuit for compressive sensing 

signal reconstruction", IEEE Trans. Info. Th., May 2009. 
[22] D. Needell and J. A. Tropp, "Cosamp: Iterative signal recovery from 

incomplete and inaccurate samples", Appl. Comp. Harm. Anal, 2008. 



